Request processing techniques

ABSTRACT

A computer system implements a hypervisor which, in turn, implements one or more computer system instances and a controller. The controller and a computer system instance share a memory. A request is processed using facilities of both the computer system instance and the controller. As part of request processing, information is passed between the computer system instance and the controller via the shared memory.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application incorporates by reference for all purposes the fulldisclosure of U.S. patent application Ser. No. 13/964,941, filed Aug.12, 2013, entitled “APPLICATION BOOT IMAGE” and U.S. patent applicationSer. No. 13/964,977, filed Aug. 12, 2013, entitled “PER REQUEST COMPUTERSYSTEM INSTANCES”.

BACKGROUND

The servicing of electronic requests can require varying amounts ofresources. Request service, for instance, can range in scale from tiny,stateless computations to long-running massively parallel applications.The servicing of requests often requires only a limited amount ofcomputing resources, often much less than the computer systems used toservice the requests have available. As a result, computing resourcesoften go underutilized and, generally, conventional techniques forprocessing requests have numerous inefficiencies. Virtualization, inmany regards, has improved the way computing resources are utilized by,for instance, allowing a single physical computer system to implementmultiple simultaneously operating virtual computer systems, therebyproviding resizable capacity that makes it easy for a developer toelastically scale upwards.

Conventional virtualization techniques, however, are subject tofundamental limitations on the ability of a developer to scale computedownwards due to the resources required to service a request and theamortization of costs for spinning up and tearing down a virtualcomputer system (instance). Practical implementations of servicevirtualization generally rely on an expectation that the workload willhave a tenancy of minutes, hours, or even longer. For example, with manyapplications, a virtual computer system may be used relativelyinfrequently. To have the virtual computer system able to servicerequests, however, the virtual computer system must be maintained in anoperational state, which requires computing resources for the computersystem's operating system and other resources (e.g., network resources).When such computer systems are underutilized, at least some of resourcesallocated to those computer systems are generally unavailable for otheruses.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIG. 1 shows an illustrative example of a diagram illustrating variousaspects of the present disclosure;

FIG. 2 shows an illustrative example of an environment in which variousembodiments can be implemented;

FIG. 3 shows an illustrative diagram comparing general purpose computersystem instances with request instances;

FIG. 4 shows an illustrative example of a configuration of a computersystem that may be used to implement various embodiments of the presentdisclosure;

FIG. 5 shows an illustrative example of an environment in which variousembodiments can be implemented;

FIG. 6 shows an illustrative example of a process for processing arequest in accordance with at least one embodiment;

FIG. 7 shows an illustrative example of a process for processing arequest in accordance with at least one embodiment;

FIG. 8 shows an illustrative example of a process for processing arequest in accordance with at least one embodiment;

FIG. 9 shows an illustrative example of an environment in which variousembodiments can be implemented;

FIG. 10 shows an illustrative example of a process for building anapplication image in accordance with at least one embodiment;

FIG. 11 shows an illustrative example of a process for identifying asafe point in application code execution in accordance with at least oneembodiment;

FIG. 12 shows an illustrative example of a process for identifying asafe point in application code execution in accordance with at least oneembodiment;

FIG. 13 shows an illustrative example of a worker hypervisor andcomponents thereof in accordance with at least one embodiment;

FIG. 14 shows an illustrative example of a process for processing arequest in accordance with at least one embodiment;

FIG. 15 illustrates an environment in which various embodiments can beimplemented.

DETAILED DESCRIPTION

In the following description, various embodiments will be described. Forpurposes of explanation, specific configurations and details are setforth in order to provide a thorough understanding of the embodiments.However, it will also be apparent to one skilled in the art that theembodiments may be practiced without the specific details. Furthermore,well-known features may be omitted or simplified in order not to obscurethe embodiment being described.

Techniques described and suggested herein relate to processing requestsby virtual computer systems that, for the purpose of computationalefficiency, reduced latency and/or other advantages, lack several layersfrom a traditional application stack. Such virtual computer systems areimplementable using substantially fewer resources, thereby allowing fortechnical advantages such as quicker boot times, reduced computationaloverhead and the resulting ability to implement a greater number ofvirtual machines using a physical computing device (host computersystem).

In an embodiment, a hypervisor is operated by a computer system for thepurpose of virtualizing the various computing resources of the computersystem, such as processing resources, memory resources, networkresources and the like. While a hypervisor that virtualizes physicalcomputing resources of an underlying computer system is used throughoutfor the purpose of illustration, the hypervisor may be one of severalvirtualization layers and, as such, may virtualize computing resourcesthat, from the perspective of the hypervisor, are virtualizationsprovided by another hypervisor. Generally, the hypervisor may be used toimplement on the computer system a worker controller and one or morevirtual computer system instances (instances). A particular instance maylack various layers of an application stack that are traditionally usedto implement instances. For example, the instance may lack an operatingsystem and associated resources. In some examples, the instance executesa language runtime directly on the hypervisor without variouscomponents, such as an operating system kernel, various processes, alanguage virtual machine, a complete network stack and/or othercomponents. The worker controller may be implemented as a processexecuting in the hypervisor or otherwise such that the worker controllerhas access to privileged functions that the instance may lack. In someembodiments, the instance may lack an implementation of one or morelayers of a protocol stack (e.g., levels one through six of the OpenSystems Interconnection (OSI) model protocol stack) while the controllerhas an implementation of the one or more layers. Further, the controllerand the instance may each implement at least one shared layer of theprotocol stack (e.g., both the controller and the instance may have animplementation of layer seven, the application layer, of the OSIprotocol stack). The one or more shared layers may be used to transfer,using a shared memory region discussed below, information between thecontroller and the instance.

In an embodiment, the instance utilizes various components of the workercontroller for the purpose of processing various requests. To enablesuch utilization of the worker controller's capabilities, the hypervisormay implement a shared memory region shared by both the instance and theworker controller. As discussed in more detail below, data involved inprocessing a request (e.g., data in a response to a request) may betransferred to the worker controller via the shared memory region.Various techniques may be utilized to enable efficient processing ofrequests and tracking data passed through the shared memory region.

In an embodiment, executed application code of the instance makes anHTTP request using an HTTP object model. The application code mayinstantiate and configure an HTTP request object for sending orreceiving an HTTP request. While various techniques are described inconnection with HTTP, the various embodiments of the present disclosureare adaptable for use in other protocols, such as file transfer protocol(FTP), domain name service (DNS), and other protocols. In thisparticular illustrative example, the HTTP object model may access aparavirtual HTTP driver that enables communications between the instanceand the worker controller through the hypervisor. While a paravirtualHTTP driver is used for the purpose of illustration, generally,notifications from the instance to the controller of data in the sharedmemory region may be made by interaction with a paravirtual devicedriver.

As a result of the HTTP object mode accessing the paravirtual HTTPdriver, the paravirtual HTTP driver may create a request record in acontrol space within a shared memory region shared by the instance andthe worker controller. The request record may include an identifiersuitable for uniquely identifying the request record. The paravirtualHTTP driver may allocate a request slot within the shared memory regionassociated with the request record. At this point, the paravirtual HTTPdriver may make a request service hypercall using a hypercall interface.The HTTP driver may make a hypercall indicating that a request recordneeds to be serviced. As a result, the hypercall interface may notify anHTTP hypercall handler operating inside the worker controller. The HTTPhypercall handler may, in turn, retrieve the request record from theshared memory region. The request may be used by the HTTP hypercallhandler to make a native HTTP request based at least in part on theretrieved request record and a request data placed into the request slotassociated with the request record by the paravirtual HTTP driver.

The hypercall handler may construct a second HTTP request using a secondHTTP object model. The hypercall handler may access the request slotassociated with the request record and construct an entity body orconfigure the HTTP request based on contents of the request slot. Onceconstructed, the HTTP hypercall handler may make the native HTTP requestusing a native network stack. The hypercall handler may return data orresults from the second HTTP request by updating the request record andassociated request slot. In this manner, the instance is able to utilizethe network stack of the worker controller without the need to maintainits own network stack.

FIG. 1 shows an illustrative diagram illustrating various aspects of thepresent disclosure, including those discussed above. In particular, FIG.1 shows a hypervisor 100 that implements one or more instances 102,which may have limited capabilities, such as described above. To enablean instance 102 to process requests submitted to the instance, theinstance may utilize various capabilities of a worker controller 104also implemented by the hypervisor. The hypervisor may implement sharedmemory 106 that is accessible to both the instance 102 and the workercontroller 104. Data may flow in both directions, as appropriate forparticular requests being serviced by the instance. The instance may beconfigured to put data into the shared memory 106 in a manner thatallows the worker controller to access the data and complete processingof the request. Similarly, the worker controller may be configured tolocate data from the shared memory and process the data accordingly. Thehypervisor may coordinate communications between the instance and theworker controller to provide notifications of data placed into theshared memory. In this manner, despite being in isolated logical regionsenforced by the hypervisor, data can be translated between the requestinstance and worker controller so that the worker controller'scapabilities can be utilized.

As may be appreciated, and as previously mentioned, a physical host maybe among a plurality of servers interconnected in a distributedcomputing system and/or datacenter. FIG. 2 illustrates a distributedcomputing and/or datacenter environment 200 in which various embodimentsmay be exercised. A plurality of customer devices 202 communicates viapublic network 204 to datacenter 206. The customer devices may includeany devices capable of connecting via a public or other network to thedata center, such as personal computers, smartphones, tablet computingdevices, and the like. In an exemplary embodiment, the public networkmay be the Internet, although other publicly accessible networks (suchas mobile and/or wireless networks) are contemplated herein. Further,while a public network 204 is used for the purpose of illustration,other networks and/or combinations of networks, which are notnecessarily public, may be used. In some instances, customer devices maycommunicate to the data center 206 through a direct fiber optic or otherconnection (e.g., via a leased line to the data center 206). Thedatacenter 206 includes one or more management components, including butnot limited to a control plane 208, a gateway 210 and/or a monitoringentity 212, which are collectively connected via internal networking 214to a plurality of internal servers 216. The control plane 208 mayreceive requests to manipulate computing resources of the datacenter,such as provisioning resources, altering routing or performingmaintenance, including updates to code running on various components ofthe datacenter. The gateway 210 may filter and route traffic in and outof the datacenter, such as to and/or from the servers via the internalnetworking. The monitoring entity may receive and report informationabout the status of computing resources in the data center, such asinformation about the internal servers.

Each internal server may be shared by multiple logical machine slots218, each slot capable of running one or more applications, such asdescribed below, such as would be the case in a virtualization systemthat abstracts the hardware of a given server into a plurality ofsemi-independent execution environments. For example, each slot may haveaccess to one or more virtual processors (VCPUs). Any number of theplurality of the customer devices previously described may run anynumber of guest operating systems or guest applications withoutoperating systems in any number of slots, up to the limits of thedatacenter (whether physical, logical or externally imposed), and theslots are allocated to the customers according to one or more of severaloperational and/or business-related criteria, such as geographicalproximity, level of support and/or resources allocated to the user,server and/or slot health and/or readiness, and the like. Thus, thetechniques described at least in connection with FIG. 1 may be scaledand/or adapted to provide efficient request processing.

As discussed, various embodiments of the present disclosure employtechniques that allow for numerous technical advantages in connectionwith processing requests, such as more efficient use of computingresources and reduced latency. FIG. 3, accordingly, shows anillustrative example of various techniques that may be employed toachieve certain advantages. As illustrated in the figure, aconfiguration 302 of a general purpose virtual computer system(instance) as instantiated by a physical computer system (physical hostor physical host computer system). Also shown is an instance 304(request instance) instantiated by a physical computer system ascompared with the general purpose computer system.

As illustrated in FIG. 3, the general purpose instance is implementedusing appropriate computer hardware including one or more centralprocessing units (CPUs), volatile and/or non-volatile memory, networkinterface cards, and/or other computing resources. The hardwareinterfaces with a virtual machine monitor or hypervisor running directlyon the hardware, e.g., a “bare metal” or native hypervisor. Examples ofsuch hypervisors include Xen, Hyper-V®, and the like. Hypervisorstypically run at a higher, more privileged processor state than anyother software on the machine, and provide services such as memorymanagement and processor scheduling for dependent layers and/or domains.The most privileged of such layers and/or domains, in some embodimentsreferred to as dom0, resides in the service domain layer, which mayinclude an administrative operating system for configuring the operationand functionality of the hypervisor, as well as that of domains of lowerprivilege, such as guest domains including guest operating systemsand/or applications executing without traditional operating systems,such as described below. The guest domains may be heterogeneous (e.g.,running different operating systems and/or applications than eachother). The service domain may have direct access to the hardwareresources of the server 302 by way of the hypervisor, while the userdomains may not.

For a particular virtual computer system, an operating system (OS)kernel, such as a Linux kernel, may interact with the hypervisor for thepurpose of utilizing the various computing resources of the hardware.The OS kernel may, for instance, be configured to manage input/output(I/O) requests from one or more user processes by interacting with thevirtualized hardware provided by the hypervisor. The user processes mayimplement a language virtual machine, which may be a virtual machine(VM) implemented logically inside of the general purpose instance forthe purpose of implementing a particular corresponding programminglanguage, such as a scripting language. The language VM may allow alanguage runtime to create one or more threads to enable its operation.Application code may utilize the language runtime for its operation(i.e., the hardware may operate in accordance with both the applicationcode and the language runtime, where the application code may referencethe language runtime).

Referring now to the request instance configuration 304, the requestinstance is implemented in a manner that reduces computing resourceoverhead. In particular, as with the general purpose instanceconfiguration, the request instance is implemented using a hypervisorthat virtualizes hardware resources. However, the request instance isimplemented with the language runtime configured to be executed directlyon top of the hypervisor instead of through the stack illustrated forthe general purpose instance. In this manner, the overhead caused bymillions of lines of code (relative to the general purpose instance) canbe saved and utilized for other purposes.

FIG. 4 shows an illustration of how such a configuration may beimplemented in various embodiments. As illustrated, computer hardware402 may be virtualized by a hypervisor 404, such as a Xen hypervisor.The hypervisor 404 may be used by a privileged, administrative domain,Dom0, and an unprivileged domain, DomU. In an embodiment, the Dom0includes a native OS 406, which may include an OS kernel 408 (e.g., aLinux kernel) and native drivers 410 configured to enable applicationand OS interaction with the virtualized hardware provided by thehypervisor 404. The Native OS may support various administrativeapplications, such as request routing applications 412, read-onlyapplication storage 414, and an HTTP stack 416. As discussed below, theHTTP stack may be utilized by the DomU to enable operation of anapplication with less resource overhead. In some embodiments, some orall of the administrative functions may be performed by processes inseparate stub domains that are unprivileged but that are operated by aservice provider for the purpose of improving the security of the Dom0.

In an embodiment, the DomU is implemented with an application binaryinterface (ABI) to the hypervisor 404 to utilize the HTTP Stack 416. Forexample, I/O may be provided using a split driver model thatcommunicates with a real device driver stack through hypercalls. Forinstance, as noted in the figure, a Node.js HTTP module may providehttp.createServer and http.request implementation using an HTTP driverof the Dom0 rather than building a TCP/IP stack against a virtualnetwork adapter, which would require more overhead. As illustrated, inthis illustrative example, a JavaScript engine 420, virtual storagedriver 422, and HTTP driver 424 interact directly with the hypervisor404 through the ABI 418, instead of through an intervening operatingsystem. The JavaScript engine 420, virtual storage driver 422, and HTTPdriver 424 provide support for a node.js platform 426 and JavaScriptsoftware development kit (SDK) 428, which, in turn, support applicationcode 430 written in JavaScript. While JavaScript and supportingcomponents are provided herein for the purpose of illustration, thescope of the present disclosure is not limited to the embodimentsexplicitly described herein. For example, the techniques describedherein can be utilized with other scripting languages and, generally,for multiple types of application code.

FIG. 5 shows an illustrative example of an environment 500 in whichvarious embodiments can be practiced. As illustrated in FIG. 5, theenvironment includes a frontend listener 502, a request queue 504, aworker hypervisor 506 and an application image repository 508. Thecomponents of the environment 500 may be implemented using a singlephysical computer system, although some components may be implemented ondifferent computer systems that are able to communicate over a network.In an embodiment, the frontend listener is a computer system or aprocess executing thereon (i.e., instance of a computer program that isbeing executed) configured to listen for requests directed to aplurality of applications. For example, the frontend listener may bemultiplexed to listen for requests at a variety of network addresses(e.g. public and/or private Internet protocol (IP) addresses) such asdifferent hostnames, ports, and application paths. Application imagesusable to instantiate virtual computer systems may each have acorresponding network address so that, when a request addressed to anetwork address is received, the application image can be used toinstantiate a virtual computer system that processes the request.

In some embodiments, the frontend listener 502 may include programminglogic for validation to filter requests. For example, the frontendlistener may be configured with an application registry service thatindicates which network addresses correspond to valid applications. Thefrontend listener may also be configured to throttle or block improperlyformatted requests, malicious requests or requests being received inexcessive numbers. In some embodiments, the frontend listener 502 isconfigured with load balancing functionality. The frontend listener 502may, for example, hash information in the request and/or associated withthe request to determine a hash value which is used to determine whichdevice to provide the work token (e.g., to determine which request queueto use). The frontend listener 502 may also distribute work tokens usingone or more load balancing techniques, which may include distributingtokens based at least in part on the number of tokens present in each ofa number of potential request queues.

The request queue 504 may be a data structure or programming moduleutilizing a queue data structure configured to store work tokens thatcorrespond to requests received by the frontend listener and enqueued bythe frontend listener. The frontend listener 502 may, for instance, beconfigured to construct a work token corresponding to the receivedrequest. The work token may include a process identifier, slotidentifier, or other similar identifying information operable toassociate the work token with a resumption point for continued handlingof the request. The work token may include an application address basedon the listening address or on address information contained within therequest. The frontend listener may also enqueue the request work tokenby, for example, serializing the request work token to a message formatand adding the serialized token to a message queue. In some embodimentsrequest work tokens may be configured to have limited lifetimes. Forexample, the frontend listener may attach an expiration time to therequest work token as part of enqueuing the request work token on therequest queue. The request queue may be configured to automaticallyterminate or eject the request work token if the request is notsatisfied within the expiration time.

The worker hypervisor 506, in an embodiment, is a hypervisor configuredwith the ability to instantiate request instances for the purpose ofprocessing received requests. The worker hypervisor may operate within acompeting consumer environment of a plurality of worker hypervisors. Toperform its operations, the worker hypervisor 506 may be configured witha worker controller 510, which may be a process configured to processwork tokens from the request queue 504. The worker controller may beimplemented by a computing device different from a computing device thatimplements the frontend listener 502. The worker controller may beimplemented in a privileged domain of the hypervisor (whereas anyrequest instances implemented by the hypervisor may be implemented inless privileged/unprivileged domains.) Further, while the presentdisclosure uses a worker controller 510 for the purpose of illustration,the functions of the worker controller 510 may be distributed amongmultiple different processes. In other words, the worker controller 510may refer to a collection of multiple processes. Generally, componentsillustrated herein, unless otherwise clear from context, can beimplemented in various ways (e.g., by distributing responsibility forvarious functions among multiple different processes) and the scope ofthe present disclosure is not necessarily limited to the illustrativeembodiments described explicitly herein. Returning to the illustrativeexample of FIG. 5. To dequeue a request from the request queue 504, theworker controller may obtain an exclusive, limited-time lease to therequest work token without removing the request work token from therequest queue. The request work token may automatically become availableagain or return to the request queue if the worker controller does notsatisfy the request within a limited amount of time. In someembodiments, the worker controller 504 is configured to regulate thenumber of request tokens currently being processed. The workercontroller 504 may, for instance, dequeue work tokens upon detectingavailable capacity for processing requests, while avoiding dequeuingwork tokens while lacking additional capacity for request processing.

As another example, the worker controller 510 may be configured toretrieve an appropriate application image to perform the request from arepository of application images 508. The worker controller 510 may, forinstance, determine an appropriate application image based at least inpart on the request work token. For example, in some embodiments, theworker controller may parse the application address within the requestwork token as a URI, extract a portion of the URI for a request path,and consult a directory service to lookup an application image for therequest path. In some embodiments, the worker controller 510 may consulta cache of application images (not pictured) already available at theworker hypervisor prior to accessing an external application imagerepository. The cache may be configured to enable faster access toapplication images than the application image repository 508. The cachemay, for instance, be implemented in random access memory (RAM) whereasthe application repository may utilize slower but more persistentstorage, such as a hard drive with spinning magnetic media, a solidstate drive or other device.

In various embodiments, the worker controller 510 is also configured toinstantiate a request instance 512. The worker controller 510 may, as anexample, be configured to interact with the worker hypervisor 506 toperform various operations such as, constructing a new user partitiondedicated to the request instance, allocating processor, memory, orother resources to the user partition using a control applicationprogramming interface (API) on the worker hypervisor 506, constructing ashared memory region and directing the worker hypervisor 506 to map theshared memory region into the address space of the user partition asread-only memory. The worker controller 510 may also interact with abootstrap program that is configured to copy at least a portion of theapplication image from the shared memory region into memory allocated tothe user partition. The bootstrap program may, for instance, retrieve anentry point address (e.g., the address to which an instruction pointerpointed at the time of the snapshot used for the application image)associated with the copied portion of the application image from theshared memory region and may begin executing application code based onthe entry point.

In addition, as noted above, the worker controller can utilize a worktoken to establish a logical connection with the corresponding requestreceived by the frontend listener 502. In this manner, when applicationcode in the request instance attempts to access the request, the workercontroller 510 may locate the request work token associated with therequest instance and establish a connection to the frontend listeneridentified by the request work token. The frontend listener, in turn,may be configured to listen for work connection requests from therequest instance. The frontend listener may, for instance, be configuredto locate the received request based on the identifying informationincluded in the request work token. The frontend listener may also beconfigured to duplicate a socket handle used to receive the receivedrequest and may give the duplicated socket to the listener listening forwork connection requests. The listener listening for work connectionrequests may read and write data using the duplicated socket inaccordance with the request instance 512.

FIG. 6 shows an illustrative example of a process 600 that may be usedto respond to a request. Operations of the process 600 may be performed,for instance, by various components of the environment 500 discussedabove in connection with FIG. 5, as discussed in more detail below. Inan embodiment, the process 600 includes receiving 602 a request, whichmay be a request submitted by a user device over a network, such asdescribed above. The request may be received, for instance, by afrontend listener, such as described above in connection with FIG. 5. Arequest instance may then be instantiated 604 to process the request. Aworker controller of a worker hypervisor may, for instance, access anapplication image from a cache or application image repository.

Once an appropriate request instance has been instantiated 604, theprocess 600 may include providing 606 request data to the instantiatedrequest instance. The request data may include data to be processed byfulfilling the request and any metadata needed by the application of theinstantiated request instance for processing the request. It should benoted that the request data may not be entirely contained in therequest. For example, a request may be configured to initiate streamingof data, where the amount of data may be too large to fit within therequest itself. In such instances, the request data may be provided in astreaming process to the request instance. Various techniques forproviding request data to a request instance are discussed in moredetail below.

Once the request instance has processed the request data that it wasprovided 606, the process 600 may include receiving 608 a response fromthe request instance. As discussed in more detail below, the requestinstance may place a response in a shared memory region shared both bythe request instance and the worker controller. Further, as with therequest data, data for a response may not be entirely contained in asingle communication, but may involve more complex types of datatransfer, such as streaming. Once the response to the request has beenreceived 608, the process 600 may include providing 610 the response tothe requestor (i.e., to the computer system that originally submittedthe request) which may involve transmission of the response over anetwork to the requestor.

In various embodiments, when a request processing is completed by arequest instance, the request instance may not be needed until anotherrequest satisfiable by the request instance is received. Accordingly, asillustrated in FIG. 6, the process 600 may include detecting 612completion of request processing. For example, in some embodiments,receipt of the received response and/or dispatch of the response to therequestor triggers completion of request processing. In someembodiments, the request instance may remain operational for an amountof time which may end in various ways, such as by notification by theapplication operating in the request instance, the expiration of a timerand/or in other ways. In some examples, for instance, it may bedesirable to leave a request instance operational to handle additionalrequests. Regardless of what triggers completion of request processing,the process 600 may include deinstantiation (i.e., deconfiguring) 614the request instance so that computing resources reserved for therequest instance become available for other purposes, such as forimplementing additional request instances.

FIG. 7 shows a more detailed example of a process 700 for processingrequests in accordance with various embodiments. As with the process 600described above in connection with FIG. 6, the process 700 may beperformed by components of the environment 500, such as described inmore detail below. In an embodiment, a request may be received 702, suchas by a frontend listener. As indicated by the arrow looping from thebox 702, a component that received the request may receive multiplerequests before the processing of a given request has completed.Operations of the process 700 or a variation thereof may be performedfor each such received request.

When a request is received 702, a determination may be made 704 whetherto process the request. As noted above, a frontend listener thatreceives the request (or another component) may be configured with anapplication registry service that indicates which network addressescorrespond to valid applications. As another example, the frontendlistener or another component may be configured to throttle or blockimproperly formatted requests, malicious requests, or requests beingreceived in excessive numbers. Accordingly, determining whether toprocess the request may be performed based at least in part on anycriteria implemented by the frontend listener or another componentoperating in connection with performance of the process 700.

If determined 704 to process the request, the process 700 may includeconstructing 706 a request work token. The work token may, for example,be constructed by a frontend listener to include various informationthat enables association of the work token with the correspondingrequest. The work token may be constructed, for example, to include aprocess identifier, slot identifier, or other identifying informationusable to associate the work token with a resumption point for continuedhandling of the request. The work token may also be constructed toinclude an application address based on the listening address or onaddress information contained within the request. Generally, the typeand amount of information in a work token may vary in accordance withthe various embodiments. It should be noted that, in some embodiments,the work token may be constructed asynchronously (e.g., before therequest is received 702). The work token may be, for instance,pre-generated and associated with a request that is received and/or withother relevant information about the request.

Once constructed (or otherwise associated with the request), the process700 may include enqueueing 708 the work token into a request queue, suchas described above. The process 700 may also include dequeuing 710 awork token, which may be the oldest work token in the request queue,which may or not be the same work token discussed above (although, asdiscussed above, the work token that was enqueued as discussed above mayeventually be dequeued in accordance with repetition of portions of theprocess 700). The work token may be dequeued by a suitable component ofa computer system, such as by a worker controller of a workerhypervisor. Further, as noted, dequeuing the work token may includeobtaining, from the request queue, an exclusive, limited-time lease tothe request work token without removing the request work token from therequest queue so that the request work token may become available againif the worker controller does not satisfy the request within a limitedamount of time (e.g., if the computer system implementing the workcontroller malfunctions during processing of the request). Adetermination may be made 712 whether the dequeued token is valid. Forexample, as discussed above, a token may include information indicatingan expiration. Determining 712 whether the token is valid may includecomparing an expiration time with a current time, where the token may beconsidered as invalid if the expiration time is before the current time.Other criteria may be checked in determining whether the token is valid.

If determined 712 that the token is valid, the process 700 may includedetermining 714 an application image appropriate for processing therequest. For example, determining 714 the application image may be basedat least in part on information associated with the token. The tokenmay, for instance, include an identifier of an appropriate applicationor the application may otherwise be determined from the information ofthe request token. For example, determining the application may includeparsing the application address within the request work token as auniform resource identifier (URI), extracting a portion of the URI for arequest path, and consult a directory service to look up an applicationimage for the request path. As another example, information from thework token may be used to look up the appropriate application in a tableor other data structure stored externally to the token and perhapsaccessible over a network (e.g., via a web service request). Generally,any method of determining the application from the work token may beused.

When an appropriate application has been determined 714, the determinedapplication may be retrieved 716. For example, the application image maybe obtained from a local cache, a local data storage device, or anexternal location. In some embodiments, a worker controller checks alocal cache for the application and, upon a cache miss, retrieves theapplication image from another location, such as from an externalapplication image repository. Upon accessing the application image (orat least a portion thereof suitable for beginning processing therequest), the process 700 may include using the retrieved applicationimage to instantiate 718 a request instance, which may be configuredsuch as described above in connection with FIGS. 3 and 4. For example,the worker controller may direct the worker hypervisor to instantiate arequest instance based at least in part on the application image. Forexample, a worker controller may direct a worker hypervisor to constructa new user partition dedicated to the request instance and allocateprocessors, memory, or other resources to the user partition using acontrol API on the worker hypervisor. The worker controller may alsoconstruct a shared memory region including at least the applicationimage and may direct the worker hypervisor to map the shared memoryregion into the address space of the user partition as read-only memory.Further, the worker controller may configure the user partition with abootstrap program. The bootstrap program may be configured such that,when executed, at least a portion of the application image from theshared memory region is copied into memory allocated to the userpartition. The bootstrap program may retrieve an entry point addressassociated with the copied portion of the application image from theshared memory region and may begin executing application code based onthe entry point.

Upon instantiation 718 of the request instance, the request instance maybe caused to process 720 the request. For example, the request instancemay establish a connection with the request using the request worktoken, to obtain data from the request for processing, such as describedin more detail below.

As illustrated in FIG. 7, the process 700 may include determining 722whether there are additional work tokens to process, such as by queryinga request queue or monitoring whether a notification of another tokenhas been received from a request queue (if so configured to providenotifications). If it is determined that there is at least one or morework token in the request queue, the process 700 may include dequeueing710 another work token and using the work token to process anotherrequest, such as described above. As indicated in the figure, if it isdetermined 722 that there are no additional tokens, the process 700 mayinclude continuing to monitor for received work tokens so that futurerequests can be serviced.

When a request is received and it is determined 704 to not process therequest or it is determined 712 that a token corresponding to thereceived request is not valid, the process 700 may include denying 724the request. Denying the request may be performed in any suitablemanner, such as by transmitting a message indicating the denial and/or areason for the denial or by simply not providing a response.

FIG. 8 shows an illustrative example of a process 800 for processing arequest, in accordance with various embodiments. The process 800 may beperformed by any suitable system, such as one or more components of theenvironment 500 discussed above in connection with FIG. 5, as discussedin more detail below. In an embodiment, the process 800 includesdirecting 802 a worker hypervisor to instantiate a request instancebased at least in part on an application image. Directing 802 the workerhypervisor to instantiate the request instance may be performed, forexample, as a result of having received a request that is serviceable bythe application for which the application image was created. At somepoint after instantiation of the request instance, the application codeof the application image is executed and execution may include anattempt to access the request for processing the request. Accordingly,the process 700 may include detecting 804 the application code's attemptto access the request (i.e., an attempt by a computer system executingthe application code). As a result of detecting 804 the attempt toaccess the request, a worker controller may locate 806 (e.g., accessfrom memory, perhaps selecting from multiple work tokens being stored)the corresponding work token associated with the request instance. Theworker controller may then use the located work token to establish 808 aconnection between the request, being stored by a frontend listener, andthe work token. The connection may be established, for instance, atleast in part by storing in memory an association between the work tokenand the request.

Once the connection has been established 808, the process 800 mayinclude using 810 the token to locate the request and providing 812 datato the request instance. For example, the frontend listener may belistening (i.e., monitoring) for work connection requests from theworker controller and, when such a work connection request is received,may locate the received request based on the token provided from theworker controller and/or information derived therefrom. The frontendlistener may duplicate a socket handle used to receive the receivedrequest and may give the duplicated socket handle to the listenerlistening for work connection requests. The listener listening for workconnection requests may read and write data using the duplicated socketin accordance with the request instance. Once the data has been provided812 to the request instance, a response may be received 814 from therequest instance, such as described in more detail below. The responsemay then be provided 816 to the requestor (i.e., the computer systemwhose request triggered performance of the process 800).

As noted elsewhere herein, numerous variations of the embodimentsexplicitly described herein are considered as being within the scope ofthe present disclosure. For example, the above illustrates embodimentswhere a worker controller determines and obtains application imagesappropriate for instantiating virtual computer systems for processingrequests. Other entities in various computing environments may performsuch functionality. For example, the frontend listener or another systemmay determine which application image to access and provide informationidentifying the image or a location where the image can be found (e.g.,by a URI). Similarly, the frontend controller may itself obtain theapplication image and transfer the application image to the workercontroller. Further, the above embodiments include those where tokensare enqueued into a request queue that is processed by a workercontroller. Generally, a frontend listener can notify a workercontroller in a variety of ways, such as by pushing notifications to theworker controller that cause the worker controller to process logic thatenables the request instance to obtain the request. Other variations arealso considered as being within the scope of the present disclosure.

As discussed above, application images are generated to be able toinstantiate virtual computer systems (request instances) for thepurposes of servicing received requests. Various techniques of thepresent disclosure relate to the generation of such application imagesto enable efficient instantiation to provide lower latency responding torequest while utilizing lower amounts of computing resources. FIG. 9,accordingly, shows an illustrative example of an environment 900 inwhich various embodiments of the present disclosure may be practiced. Inthe example environment 900 of FIG. 9, an application source 902, buildsystem 904, build hypervisor 906, and worker hypervisor 908. Theapplication source may be provided by any system or component thereofthat provides application code to be used in building an applicationimage. The application code may have been received by a developer, suchas a developer of a customer of a computing resource service provideror, generally, any developer able to contribute application code for thepurpose of building application instances. The application source may besubmitted by a developer through, for example, a web service applicationprogramming interface (API) or a version control system. The applicationsource may include source files, configuration files, resource files(such as web pages, images, or other media files), binary files and/orthe like.

The build system 904 may be a system or component thereof configured, asdescribed in more detail below, to operate as a pre-processor forapplication code before it is processed into an application image, andplaced into an application image repository 910. The worker hypervisormay be a worker hypervisor such as described above, e.g., configuredwith one or more processes that access an application image from theapplication image repository 910 and use the accessed application imageto instantiate a request instance 912 for processing a request. Thevarious components of FIG. 9 may be implemented on separate physicalcomputer systems, although various embodiments of the present disclosureinclude embodiments where the same physical computer system implementsboth the build hypervisor 906 and the worker hypervisor 908, such aswith an additional layer of virtualization.

In an embodiment, the build system 904 is configured to processapplication source using a variation aware parser 914 to produce anannotated source. The variation aware parser may be a system orcomponent thereof (e.g., process executing on a system) configured toexamine an executable portion of the application source (e.g., sourcecode and/or compiled source code) to determine one or more locations(variation points) at which execution may first begin to vary in theprogram. Examples of variation may include execution reliant on receiptof a network message, reading user input, reading the system clock,utilization of a random number generator and/or performing other similaractions that may result in accessing information not deterministicallyderivable from the application source. In other words, a variation pointmay correspond to one or more computer executable instructions whoseresults of execution potentially varies among multiple executions. Acatalogue of functions whose invocation can cause a variation inapplication execution may be maintained to enable the variation awareparser to identify locations in the execution where execution may beginto vary.

The variation aware parser 914 may also be configured to placeannotations in the application source or may store annotations inmetadata associated with the application source to record the one ormore determined locations. In some embodiments, the variation awareparser is configured to apply static analysis of the program's structureto first identify one or more entry points for executing the applicationsource. The variation aware parser 914 may, for instance, parse,interpret, or otherwise analyze the application source beginning fromthe one or more entry points until a potential variation is detected.The variation aware parser 914 may store the determined variationlocations by, for example, recording a list of the lines of source codeor executable machine instructions corresponding to the determinedlocations. In some embodiments the variation aware parser is configuredto interact with annotations placed by the developer in the applicationsource. For example, the variation aware parser may read and processannotations that override whether a program location should or shouldnot be considered a variation. Such developer annotations may be inaccordance with a syntax that the variation aware parser 914 isconfigured to process.

In some embodiments, the build system is configured to transmit theannotated application source to a build controller 916 implemented bythe build hypervisor 906, where the build controller 916 may be aprocess being executed on the build hypervisor 906. In an embodiment,the build system 904 selects the build hypervisor from among a pluralityof hypervisors ready to receive build requests. The build system 904 maypackage the annotated source in an archive format and store the archivein a location available to the build controller. The build system 904may communicate with the build controller 916 to initiate an applicationbuild, such as by making a web service request to the build controller,including the archive location, to initiate the build request.

In an embodiment, the build controller 916 is configured to access abuild bootstrap program. The build bootstrap program may be stored in arepository 920 of bootstrap programs, which may be on the same physicalcomputer system as the build hypervisor 906 or may be in anotherlocation within a distributed system and, as a result, accessible over anetwork. The build controller may analyze the annotated source todetermine an appropriate bootstrap program for the application. Forexample, the build controller may analyze the annotated source forplatform requirements or other factors that may influence the selectionof a bootstrap program. There may be different bootstrap programs fordifferent application types. There may be a bootstrap program forJavaScript applications, another bootstrap program for Rubyapplications, and so on. Further, each type of application (JavaScript,Ruby, etc.) may have multiple bootstrap programs from which to select,each of which may be appropriate for one or more particular sub-types ofapplications. In some embodiments bootstrap programs are each configuredto boot a kernel for the annotated source. The bootstrap program may,when executed, operate in accordance with routines to read the kerneland annotated source into memory. The bootstrap program may also includeroutines to set breakpoints at the determined variation locations in theannotated source. The bootstrap program may further include routines tobegin executing the kernel from a kernel entry point.

In an embodiment, the build controller 916 creates a build instance 918based on the annotated source and build bootstrap program. The buildcontroller may, for instance, instantiate a virtual machine from whichthe annotated source and bootstrap program may be accessible. The buildcontroller may attach to the build instance a variation monitor 922operable to detect and respond to variation events to the virtualmachine. The variation monitor 922 may be a process executed on thehypervisor 906 configured to analyze execution of the application anddetect variation events. The build controller may also be configured toinstruct the instantiated virtual machine to execute the bootstrapprogram. Upon execution of the bootstrap program, the variation monitor922 may be configured to halt the build instance in response to thebuild instance reaching a variation point in the application. Forexample, the variation monitor 922 may be implemented using a virtualmachine breakpoint. To do this, the variation monitor 922 may beconfigured to receive notifications of reached breakpoints and halt thevirtual machine in response to receiving a notification. The variationmonitor 922 may utilize hypervisor functionality to instruct thehypervisor to suspend the program when the CPU is executing a particularcode instruction or when another event indicating reaching a variationpoint is detected. As one example, the variation monitor may readinstructions placed into the application source or memory pages thatcontain the application source to determine where the variation pointsare. When an instruction is encountered that would result in avariation, the variation monitor 922 may issue an interrupt whose valuecorresponds to halting the virtual machine executing the applicationcode. The hypervisor may trap the interrupt and an interrupt handler ofthe build instance's CPU may cause execution of the application to stop.

A second approach to that is to use the break point instruction which isan interrupt instruction used by systems employing an IntelArchitecture. As part of the bootstrap, an interrupt handler may beinstalled to detect the interrupts (e.g., an interrupt with value 3 or,generally, a numerical value for interrupts corresponding to a need tohalt execution). Upon detection of an interrupt, interrupt handler codemay be executed in response to that interrupt being raised within thevirtual machine. As part of the interrupt handler, a communication maybe provided to the variation monitor that indicates the trappedinstruction, thereby indicating a need to cease execution of the virtualmachine. In other words, in this approach, control is first transferredto another piece of code running inside the build instance, but thenupon communication with the variation monitor, the machine down is shutdown so that a snapshot can be taken.

As part of the build process, the build controller may be configured totake a snapshot image of the build instance that has been halted. Thebuild controller may, for instance, make a copy of the memory space ofthe virtual machine including the state of processor registers, flags,program counters, and other aspects of the virtual environment. Thesnapshot image may include an entry point address at which execution ofthe snapshot image may be resumed. In some embodiments the buildcontroller may be configured to move execution of the build instance toa nearby execution safe point. For example, the build controller mayadvance or regress the machine instruction pointer to avoid snapshottingthe image while executing certain kernel routines, critical sections orother unsafe portions of execution. The build controller may move thebuild instance to a nearby state at which some or all of the processorregisters do not need to be restored to resume execution. A snapshot maybe taken by the build controller 916 once the build instance has beenmoved to an execution safe point.

As noted above, an application image built in this manner may beutilized to instantiate a request instance (or, generally, any instancethat is based at least in part on the application image). In someembodiments, the build controller 916 is configured to place thesnapshot image in a repository of application images accessible by oneor more worker hypervisors. A worker hypervisor may construct a requestinstance based on the snapshot image by retrieving an application imagefrom the repository of application images and resume execution of theapplication at the entry point address.

FIG. 10 shows an illustrative example of a process 1000 for building anapplication image that may be used for various purposes, such as forinstantiating request instances or other instances. Operations of theprocess 1000 may be performed by any suitable system, such as by a buildsystem and/or build hypervisor, such as described above in connectionwith FIG. 9 and as described in more detail below. In an embodiment, theprocess 1000 includes receiving 1002 application source code, such asdescribed above. For instance, the application source code may bereceived from a developer that developed the source code. A variationaware parser may be used to process 1004 the received application sourcecode and potential variations in program execution may be determined andrecorded 1006. For instance, as discussed above, a build system maycause the variation aware parser to examine an executable portion of theapplication source to determine one or more locations at which executionmay first begin to vary in the program and place annotations in theapplication source. Alternatively or in addition, annotations may bestored in metadata associated with the application source to record theone or more determined locations. Determining the potential variationsmay include identifying one or more entry points for executing theapplication source and parsing, interpreting or otherwise analyzing theapplication source beginning from the one or more entry points until apotential variation is detected. Further, as noted, the variation awareparser may interact with annotations placed by the developer in theapplication source, for instance, to determine whether any annotationsoverride whether a program location should or should not be considered avariation. The determined potential variation locations may be stored inany suitable manner, such as by recording a list of the lines of sourcecode or executable machine instructions corresponding to the determinedlocations.

As illustrated in FIG. 10, the process 1000 may include selecting 1008 abuild hypervisor. A build hypervisor may, for instance, be selectedbased at least in part on its availability for generating applicationboot images. A build hypervisor may also be selected based at least inpart on one or more other factors, such as having a configurationsuitable for building applications of a particular type (e.g.,JavaScript or Ruby). Once selected, the annotated application source maybe transmitted 1010 to the selected build hypervisor for processing. Forinstance, if the application source is annotated on a different systemthan the system that operates the build hypervisor, the annotatedapplication source may be transmitted over a network to the buildhypervisor. A bootstrap program may also be selected 1012. As discussed,different types of applications may correspond to different bootstrapprograms. For example, a particular bootstrap program may be configuredfor a particular application type. In alternate embodiments, however,bootstrap programs may be configured to be more complex, such as byhaving the ability be used for multiple application types. In variousembodiments, a bootstrap program is configured to boot a kernel for theannotated source. The bootstrap program may, for instance, includeroutines to read the kernel and annotated source into memory. Thebootstrap program may include routines to set breakpoints at thedetermined variation locations in the annotated source. The bootstrapprogram may also include routines to begin executing the kernel from akernel entry point.

In an embodiment, the process 1000 includes instantiating 1014 a virtualmachine able to access both the bootstrap program and the annotatedsource code. The virtual machine may, for instance, be instantiatedunder the direction of the build controller discussed above. The buildcontroller may also attach to the instantiated virtual machine avariation monitor that is operable to detect and respond to variationevents to the virtual machine. The build controller may instruct theinstantiated virtual machine to execute the bootstrap program. Theapplication may then be executed and monitored 1016 by the variationmonitor which may halt 1018 the build instance in response to the buildinstance reaching a variation point in the application. For example, thevariation monitor may be implemented using a virtual machine breakpoint.The variation monitor may be configured to receive notifications ofreached breakpoints and halt the virtual machine in response toreceiving a notification. A snapshot of the build instance may be taken1020. In an embodiment, taking the snapshot 1020 includes the buildcontroller making a copy of the memory space of the virtual machine,including the state of processor registers, flags, program counters, andother aspects of the virtual environment. The snapshot image may includean entry point address at which execution of the snapshot image shouldresume. As discussed below, taking the snapshot may also include movingthe build instance to a nearby execution safe point. For example, thebuild controller may advance or regress a machine instruction pointer toavoid snapshotting the image while executing certain kernel routines,critical sections, or other unsafe portions of execution. The buildcontroller may also move the build instance to a nearby state at whichsome or all of the processor registers do not need to be restored toresume execution.

In an embodiment, the process 1000 includes storing the snapshot in alocation where the snapshot can be used at a later time to instantiate arequest instance or another instance. Accordingly, as illustrated inFIG. 10, the process 1000 includes placing 1022 the snapshot in anapplication image repository so that the snapshot may be used at a latertime, such as to instantiate a request instance as described below.

As with all techniques disclosed explicitly herein, variations areconsidered as being within the scope of the present disclosure. Forexample, variations in determining when to snapshot a build instanceand/or which snapshot to use may be used so that snapshots that are usedfor instantiation of an instance are reliable and instantiable withoutprocessing that may be unnecessary with proper setup.

In some embodiments, safe points in an application execution areidentified by a variation aware parser, such as described above. FIG.11, accordingly, shows an illustrative example of a process 1100 whichmay be used to determine a safe point for taking a snapshot. The process1100 may be performed by any suitable system, such as a variation awareparser or another system having such functionality. In an embodiment,the process 1100 includes accessing 1102 a first instruction anddetermining 1104 whether the instruction corresponds to a variationpoint. Determining 1104 whether the instruction corresponds to avariation point may be performed, for example, as described above. Ifdetermined 1104 that the current instruction does not correspond to avariation point, the process 1100 may repeat the operations of accessing1104 the next instruction and determining whether the accessedinstruction corresponds to a variation point. This may repeat untildetermined 1104 that the currently accessed instruction corresponds to avariation point.

When determined 1104 that the currently accessed instruction correspondsto a variation point, the process 1100 may include determining 1106whether the currently accessed instruction corresponds to a safe point.Determining 1106 whether the currently accessed instruction correspondsto a safe point may be performed in any suitable manner. For example, insome embodiments, the currently accessed instruction is analyzed todetermine any functions connected with the instruction and checkingwhether any determined functions appear in a catalogue of functions thatare identified as unsafe (e.g., because invocation of such functions canresult in machine interrupts being set at a particular state or becauseinvocation of the functions corresponds to activity that is not able tobe reconstructed in a suspend and resume process).

If determined 1106 that the currently accessed instruction does notcorrespond to a safe point, the process 1100 may include accessing 1108a previous instruction (e.g., the immediately prior instruction in asequence of instructions). A determination may be made 1106 againwhether the currently accessed instruction corresponds to a safe point.This process may repeat by successively accessing previous instructionsand determining whether they correspond to safe points until adetermination is made 1106 that the currently accessed instructioncorresponds to a safe point. When determined 1106 that the currentlyaccessed instruction corresponds to a safe point, the process 1100 mayinclude identifying 1110 the current instruction as a variation pointand safe point so that, when processed by a variation monitor (or othersuitable system), execution of the application is halted at theidentified point. In this manner, a safe point is identified as avariation point for later processing instead of the actual point ofvariation which may not be a safe point.

FIG. 12 shows an alternate process of determining a safe point usablefor constructing an application image. The process 1200 may be performedto compute safe points dynamically as part of a build instance. Theprocess 1200 may be performed by any suitable system, such as avariation monitor such as described above or other such system. In theprocess 1200, snapshotting may be initiated 1202 so that snapshots aretaken multiple times during execution of the application code by thebuild instance. Initiating the snapshotting may be performed, forexample, so that snapshots are taken periodically, such as everymicrosecond (or other amount of time) and/or with execution of everyinstruction.

In an embodiment, the process 1200 includes processing 1204 thefirst/next instruction, where processing may include executing theinstruction and/or analyzing the instruction as it is processed. Adetermination may be made 1206 whether the instruction corresponds to avariation point, such as described above. If determined 1206 that theinstruction does not correspond to a variation point, the process 1200may repeat by processing the next instruction until determined 1206 thatthe currently processed instruction corresponds to a variation point.When determined 1206 that a current instruction corresponds to avariation point, the process 1200 may include accessing 1208 the firstsnapshot in a sequence of snapshots proceeding backwards in time. Thefirst snapshot may be, for instance, the most recent snapshot takenbefore the instruction corresponding to the variation point wasencountered.

A determination may be made 1210 whether the snapshot corresponds to asafe point. The determination 1210 may be made based at least in part onan analysis of the snapshot. A set of safe point criteria may be checkedand compliance with the criteria may indicate that the snapshotcorresponds to a safe point. The criteria may be based at least in parton a state of the CPU. For example, criteria based at least in part onthe CPU register state may include whether an interrupt flag is clear orset, whether the CPU is handling an exception, whether the CPU is in themiddle of a page fault and, generally, whether the CPU register isreproducible. The criteria may also be based at least in part on whetherthe CPU is at an unsafe point caused by the application code because,for instance, the application code has acquired a critical section lock,whether the CPU, as indicated by a CPU instruction pointer, isprocessing instructions marked as unsafe. If determined 1210 that thesnapshot does not correspond to a safe point, the process 1200 mayinclude repeatedly accessing 1208 the next (moving backwards in time)snapshot and determining 1210 whether the accessed snapshot correspondsto a safe point until determined 1210 that the currently accessedsnapshot corresponds to a safe point. Any snapshots not identified assafe may be discarded, either upon determining that they do notcorrespond to a safe point, upon completion of the process 1200 orotherwise. When determined 1210 that a snapshot corresponds to a safepoint, the snapshot may be used 1212 for an application image.

Generally, the above techniques provide techniques for computing thatentry point, which is based at least in part on the position of theinstruction pointer of the CPU at the time a snapshot was taken. Thisprovides a place for resumption as part of booting the application imagedue to having available the exact state. Further, there may beadditional bootstrap code that runs before jumping to that snapshotpoint and that could be for doing things such as restoring the values ofCPU registers or other configuration information for the machine. Forinstance, when a machine boots up, its bootstrap program may reconstructthat exact state that corresponds to the snapshot image. Because afacility in a hypervisor may not make a perfect copy, but may haveinformation about the state in memory, the bootstrap program can fix anyimproper state if the bootstrap program is provided with additionalstate information. The additional state may information may beaccessible to the bootstrap program as metadata or supplementaryapplication image. The bootstrap program may use the additional stateinformation to restore application state before jumping into the entrypoint of the application.

As noted above, various embodiments of the present disclosure reduce thecomputational overhead required for an instance instantiated inaccordance with the various techniques described herein. As a result,instance provisioning is able to proceed quickly and with minimal wastedresources. FIG. 13 shows an illustrative example of an environment whichmay enable an instance, such as a request instance, to process requests.As illustrated in FIG. 13, the environment is implemented inside of aworker hypervisor 1302, which may be a worker hypervisor, such asdescribed above. The worker hypervisor 1302 may implement a hypercallinterface 1304, which may be an interface that allows a guest operatingsystem or guest application (e.g. request instance) to make requests tothe hypervisor. The hypercall interface 1304 may be configured toreceive hypercalls from a paravirtual hypertext transfer protocol (HTTP)driver 1306 in a request instance 1308 (or other instance which is notnecessarily instantiated as a result of a received request that needs tobe processed) implemented on the worker hypervisor 1302. The hypercallinterface may provide hypercalls received from the paravirtual HTTPdriver 1306 to an HTTP hypercall handler 1320. While HTTP is usedthroughout for the purpose of illustration, the various techniquesdescribed herein may be adapted to support other protocols which may beused by an instance.

The paravirtual HTTP driver 1306 may be configured to provide a systemcall interface for making HTTP requests similar to or in correspondencewith the interface for making HTTP requests provided by an HTTP objectmodel 1310. The HTTP object may translate the configured HTTP request toformat the request configuration appropriately for the system callinterface. The paravirtual HTTP driver 1306 may also be configured tocreate a request record in a control space 1312 within a memory region1314 shared between the request instance 1308 and a worker controller1316, which may be as described above (e.g., the worker controllerdescribed above in connection with FIG. 5). The control space 1312 maybe a space reserved for storing information about data stored in requestrecords of the shared memory region 1314. The paravirtual HTTP driver1306, for instance, may be configured to write data to the control space1312 that indicates where data of the request is located in the sharedmemory region 1314. Information may be included may comprise anidentifier of the request slot, information that indicates the size ofthe data taken in the request slot, the size of the request slots (if anembodiment where the size may vary) and, generally, information thatallows identification of the correct amount of data from the requestslots.

The shared memory region may be implemented in various ways, such as acircular (ring) buffer. In various embodiments, each request instanceimplemented on the hypervisor is provided its own shared memory buffer,although the various embodiments described herein may be adapted so thatmultiple request instances share a memory buffer (e.g., by trackingadditional information to be able to match requests to requestinstances). The HTTP driver may be configured with a memory location atwhich the control space is located. The HTTP driver may construct arequest record within the control space based on configurationinformation from the HTTP request.

As illustrated in FIG. 13, the request instance may lack variousresources necessary for processing some requests. For example, theprocessing of various requests may include use of a network stack(protocol stack), such as a transmission control protocol/Internetprotocol (TCP/IP) stack or other network stack, where a network stackmay be a program that implements a protocol suite. The request instance1308 may lack a network stack so as to not require the overhead requiredto use the network stack. To process requests, however, the requestinstance may interact with a network stack 1318 implemented by theworker controller 1316. The worker controller 1316 may operate a singlenetwork stack that is used for multiple request instances implemented onthe worker hypervisor 1302. Accordingly, the hypercall interface 1304may be configured to process certain calls from the paravirtual HTTPdriver 1306 to a HTTP hypercall handler 1320 that may be a process ofthe worker controller 1316 configured with programming logic forresponding to hypercalls from the paravirtual HTTP driver 1306 throughthe hypercall interface 1304. The HTTP hypercall hander 1320 may beconfigured to direct communications involved in processing requestsusing the network stack 1318. As discussed in more detail below, theactual data for the request may be provided to the worker controllerthrough the shared memory region 1314 using one or more request slots1322 of the shared memory region 1314. For example, when applicationcode 1324 is executed, data to and from the application may betransmitted via the shared memory region 1314. Data written to thecontrol space 1312 by the paravirtual HTTP driver 1306 may indicate oneor more request slots 1322 where data for the request can be located sothat the HTTP hypercall handler 1320 can read the control space 1312 todetermine which control slot(s) to obtain the data from. In this manner,the hypercall interface 1304 can be configured to handle small amountsof data (which is efficient due to the limited resources required tomanage smaller amounts of data) while still allowing relevant data of alarger size to be passed from the request instance 1308 to the workercontroller 1316.

FIG. 14 shows an illustrative example of a process 1400 for processing arequest, in accordance with an embodiment. The process 1400 may beperformed by a hypervisor (i.e., by a computer system in accordance withexecutable code for implementing a hypervisor), such as a workerhypervisor as described above. The process 1400 uses HTTP as an example,but as noted above, the process 1400 may be adapted for use with otherprotocols. In an embodiment, the process includes receiving 1402 an HTTPrequest from an application. The request may be made using an HTTPobject model. Further, the application code may instantiate anappropriate HTTP request object for responding to the request. Therequest object may be, for instance, a JavaScript XmlHttpRequest objector a Node.js createServer object. Execution of the application code mayconfigure the HTTP request with a location, parameters, data and/or thelike.

With the HTTP object in place, the application code may utilize the HTTPobject to make a system call to a paravirtual HTTP driver. Accordingly,the process 1400 may include receiving 1404 a system call to aparavirtual HTTP driver. For instance, the HTTP object may make a kernelsystem call to access the paravirtual HTTP driver which may provide asystem call interface for making an HTTP request similar to or incorrespondence with the interface for making HTTP requests provided bythe HTTP object model. The HTTP object may translate the configured HTTPrequest to format the request configuration appropriately for the systemcall interface.

Upon receipt 1404 of the system call, the paravirtual HTTP driver maycreate a request record in a control space within a shared memoryregion, e.g., a memory region shared between a worker controller and arequest instance, such as described above. The paravirtual HTTP drivermay be configured with a memory location at which the control space islocated. The paravirtual HTTP driver may create 1406 a request recordwithin the control space based on configuration information from theHTTP request. In an embodiment, the paravirtual HTTP driver maydetermine a memory block size based on the HTTP request configuration.The HTTP driver may attempt to allocate a memory block from the controlspace based on the determined size and copy an HTTP requestconfiguration to the allocated memory block. In some embodiments, theHTTP driver attaches a pointer to the allocated memory to a list ofactive HTTP request records using a lock-free compare-and-swap linkedlist implementation. The request record may include a sequence number,random value and/or other identifier suitable for uniquely identifyingthe request record. In some embodiments the identifier may be operableto further correlate the request record with one or more of: the HTTPrequest, the HTTP request object or the application code making the HTTPrequest.

In an embodiment, the process 1400 includes allocating 1408 a requestslot within the shared memory region that is associated with the requestrecord. For example, in some embodiments, the shared memory region mayinclude one or more fixed-size buffer regions organized using a ringbuffer. The HTTP driver may allocate one of the buffer regions usingmutual exclusion, such as by performing compare-and-swap assignment of alease record. The HTTP driver may associate the request slot with therequest record by setting fields in the request record to the addressand size of the request slot. In one embodiment the one or morefixed-size buffer regions may comprise a plurality of buffer sizes, eachbuffer size corresponding to a list of one or more buffer regions ofthat size. For example, a first particular buffer size may be used forservicing receive requests and a second particular buffer size may beused for servicing send requests.

Under the direction of the HTTP object model, the paravirtual HTTPdriver may make a request service hypercall using a hypercall interface.The HTTP driver may make a hypercall indicating that a request recordneeds to be serviced. The hypercall may include an identifier for therequest record so that a hypercall handler can identify the requestrecord in the shared memory region. Accordingly, the hypercall interfacemay receive 1410 the request and notify 1412 the HTTP hypercall handlerof the hypercall, thereby indicating to the hypercall handler that therequest needs servicing. In some embodiments, a worker controllerimplementing the HTTP hypercall handler is configured to receivenotifications of relevant hypercalls, such as by subscribing to ahypercall event dispatcher. The worker controller may be configured todispatch a received notification to a hypercall handler for servicingHTTP requests.

Upon receipt of the hypercall, the hypercall handler may retrieve 1414the request record from the shared memory region (e.g., by using anidentifier of the request record). For example, the hypercall handlermay walk a linked list of active HTTP requests to find a request recordwhose identifier matches an identifier provided as a hypercallparameter. Once the HTTP hypercall handler retrieves 1414 the requestrecord, the hypercall handler may build a native HTTP request based atleast in part on the retrieved request record and a request data locatedin the request slot associated with the request record. The hypercallhandler may construct a second HTTP request using a second HTTP objectmodel, such as an object model using a Portable Components (POCO) C++library or the libcurl library. For example, the hypercall handler maybuild 1416 an HTTP request using the second HTTP object model andconfigure the HTTP request using a location, parameters, or othersimilar data included in the request record. The hypercall handler mayaccess the request slot associated with the request record and constructan entity body or configure the HTTP request based on contents of therequest slot.

With the HTTP request built 1416 by the hypercall handler, the HTTPhypercall handler may make 1418 the native HTTP request using a nativenetwork stack implemented by the worker controller. The hypercallhandler may return data or results from the second HTTP request byupdating the request record and associated request slot. In someembodiments, the HTTP driver waits on a semaphore included in therequest record. The hypercall handler may update the request record andcontents of the request slot based on the second HTTP request. Thehypercall handler may signal the semaphore when updates have beencompleted. The HTTP driver may reset and wait again on the semaphoreonce the update has been processed, such as if the update represents aportion of an HTTP response stream. Alternatively, the HTTP driver maymake a new hypercall request once the update has been processed torequest further updates.

As an illustrative example of how the process 1400 may be used, the HTTPhypercall handler, upon notification of such a request from thehypercall interface, may create an HTTP listener on the native networkstack. The HTTP listener may wait for an inbound HTTP request and, whendata received for the inbound request is received, the HTTP hypercallhandler may use the shared memory to marshal data to the requestinstance. The HTTP hypercall handler may, for instance, notify theparavirtual HTTP driver of the received data which may then obtain thedata by obtaining the data from the shared memory, such as by usinginformation written by the HTTP hypercall handler to a control space tolocate one or more request slots in which the data is contained. Thedata may then be provided to the application code for any processingthat may occur.

The process 1400 may be adapted for processing various types ofrequests. For example, for streaming data, the first chunk of receiveddata may be processed such as described above. A similar process mayoccur, where communications through the hypercall handler indicate aneed to receive additional data for an existing request. The data may bepassed through the shared memory, such as described above. Similarly,for sending data, a notification may be sent through a hypercall handlerthat data is to be sent. The worker controller, upon receipt of thenotification, may obtain data placed by the instance into the sharedmemory, build a native HTTP request, and transmit the data.

FIG. 15 illustrates aspects of an example environment 1500 forimplementing aspects in accordance with various embodiments. As will beappreciated, although a web-based environment is used for purposes ofexplanation, different environments may be used, as appropriate, toimplement various embodiments. The environment includes an electronicclient device 1502, which can include any appropriate device operable tosend and receive requests, messages or information over an appropriatenetwork 1504 and convey information back to a user of the device.Examples of such client devices include personal computers, cell phones,handheld messaging devices, laptop computers, tablet computers, set-topboxes, personal data assistants, embedded computer systems, electronicbook readers and the like. The network can include any appropriatenetwork, including an intranet, the Internet, a cellular network, alocal area network or any other such network or combination thereof.Components used for such a system can depend at least in part upon thetype of network and/or environment selected. Protocols and componentsfor communicating via such a network are well known and will not bediscussed herein in detail. Communication over the network can beenabled by wired or wireless connections and combinations thereof. Inthis example, the network includes the Internet, as the environmentincludes a web server 1506 for receiving requests and serving content inresponse thereto, although for other networks an alternative deviceserving a similar purpose could be used as would be apparent to one ofordinary skill in the art.

The illustrative environment includes at least one application server1508 and a data store 1510. It should be understood that there can beseveral application servers, layers or other elements, processes orcomponents, which may be chained or otherwise configured, which caninteract to perform tasks such as obtaining data from an appropriatedata store. Servers, as used herein, may be implemented in various ways,such as hardware devices or virtual computer systems. In some contexts,servers may refer to a programming module being executed on a computersystem. As used herein the term “data store” refers to any device orcombination of devices capable of storing, accessing and retrievingdata, which may include any combination and number of data servers,databases, data storage devices and data storage media, in any standard,distributed or clustered environment. The application server can includeany appropriate hardware and software for integrating with the datastore as needed to execute aspects of one or more applications for theclient device, handling some (even a majority) of the data access andbusiness logic for an application. The application server may provideaccess control services in cooperation with the data store and is ableto generate content such as text, graphics, audio and/or video to betransferred to the user, which may be served to the user by the webserver in the form of HyperText Markup Language (“HTML”), ExtensibleMarkup Language (“XML”) or another appropriate structured language inthis example. The handling of all requests and responses, as well as thedelivery of content between the client device 1502 and the applicationserver 1508, can be handled by the web server. It should be understoodthat the web and application servers are not required and are merelyexample components, as structured code discussed herein can be executedon any appropriate device or host machine as discussed elsewhere herein.Further, operations described herein as being performed by a singledevice may, unless otherwise clear from context, be performedcollectively by multiple devices, which may form a distributed system.

The data store 1510 can include several separate data tables, databasesor other data storage mechanisms and media for storing data relating toa particular aspect of the present disclosure. For example, the datastore illustrated may include mechanisms for storing production data1512 and user information 1516, which can be used to serve content forthe production side. The data store also is shown to include a mechanismfor storing log data 1514, which can be used for reporting, analysis orother such purposes. It should be understood that there can be manyother aspects that may need to be stored in the data store, such as forpage image information and to access right information, which can bestored in any of the above listed mechanisms as appropriate or inadditional mechanisms in the data store 1510. The data store 1510 isoperable, through logic associated therewith, to receive instructionsfrom the application server 1508 and obtain, update or otherwise processdata in response thereto. In one example, a user, through a deviceoperated by the user, might submit a search request for a certain typeof item. In this case, the data store might access the user informationto verify the identity of the user and can access the catalog detailinformation to obtain information about items of that type. Theinformation then can be returned to the user, such as in a resultslisting on a web page that the user is able to view via a browser on theuser device 1502. Information for a particular item of interest can beviewed in a dedicated page or window of the browser. It should be noted,however, that embodiments of the present disclosure are not necessarilylimited to the context of web pages, but may be more generallyapplicable to processing requests in general, where the requests are notnecessarily requests for content.

Each server typically will include an operating system that providesexecutable program instructions for the general administration andoperation of that server and typically will include a computer-readablestorage medium (e.g., a hard disk, random access memory, read onlymemory, etc.) storing instructions that, when executed by a processor ofthe server, allow the server to perform its intended functions. Suitableimplementations for the operating system and general functionality ofthe servers are known or commercially available and are readilyimplemented by persons having ordinary skill in the art, particularly inlight of the disclosure herein.

The environment in one embodiment is a distributed computing environmentutilizing several computer systems and components that areinterconnected via communication links, using one or more computernetworks or direct connections. However, it will be appreciated by thoseof ordinary skill in the art that such a system could operate equallywell in a system having fewer or a greater number of components than areillustrated in FIG. 15. Thus, the depiction of the system 1500 in FIG.15 should be taken as being illustrative in nature and not limiting tothe scope of the disclosure.

The various embodiments further can be implemented in a wide variety ofoperating environments, which in some cases can include one or more usercomputers, computing devices or processing devices which can be used tooperate any of a number of applications. User or client devices caninclude any of a number of general purpose personal computers, such asdesktop, laptop or tablet computers running a standard operating system,as well as cellular, wireless and handheld devices running mobilesoftware and capable of supporting a number of networking and messagingprotocols. Such a system also can include a number of workstationsrunning any of a variety of commercially-available operating systems andother known applications for purposes such as development and databasemanagement. These devices also can include other electronic devices,such as dummy terminals, thin-clients, gaming systems and other devicescapable of communicating via a network.

Various embodiments of the present disclosure utilize at least onenetwork that would be familiar to those skilled in the art forsupporting communications using any of a variety ofcommercially-available protocols, such as Transmission ControlProtocol/Internet Protocol (“TCP/IP”), protocols operating in variouslayers of the Open System Interconnection (“OSI”) model, File TransferProtocol (“FTP”), Universal Plug and Play (“UpnP”), Network File System(“NFS”), Common Internet File System (“CIFS”) and AppleTalk. The networkcan be, for example, a local area network, a wide-area network, avirtual private network, the Internet, an intranet, an extranet, apublic switched telephone network, an infrared network, a wirelessnetwork and any combination thereof.

In embodiments utilizing a web server, the web server can run any of avariety of server or mid-tier applications, including Hypertext TransferProtocol (“HTTP”) servers, FTP servers, Common Gateway Interface (“CGI”)servers, data servers, Java servers and business application servers.The server(s) also may be capable of executing programs or scripts inresponse requests from user devices, such as by executing one or moreweb applications that may be implemented as one or more scripts orprograms written in any programming language, such as Java®, C, C# orC++, or any scripting language, such as Perl, Python or TCL, as well ascombinations thereof. The server(s) may also include database servers,including without limitation those commercially available from Oracle®,Microsoft®, Sybase and IBM®.

The environment can include a variety of data stores and other memoryand storage media as discussed above. These can reside in a variety oflocations, such as on a storage medium local to (and/or resident in) oneor more of the computers or remote from any or all of the computersacross the network. In a particular set of embodiments, the informationmay reside in a storage-area network (“SAN”) familiar to those skilledin the art. Similarly, any necessary files for performing the functionsattributed to the computers, servers or other network devices may bestored locally and/or remotely, as appropriate. Where a system includescomputerized devices, each such device can include hardware elementsthat may be electrically coupled via a bus, the elements including, forexample, at least one central processing unit (“CPU” or “processor”), atleast one input device (e.g., a mouse, keyboard, controller, touchscreen or keypad) and at least one output device (e.g., a displaydevice, printer or speaker). Such a system may also include one or morestorage devices, such as disk drives, optical storage devices andsolid-state storage devices such as random access memory (“RAM”) orread-only memory (“ROM”), as well as removable media devices, memorycards, flash cards, etc.

Such devices also can include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired), an infrared communication device, etc.) and working memory asdescribed above. The computer-readable storage media reader can beconnected with, or configured to receive, a computer-readable storagemedium, representing remote, local, fixed and/or removable storagedevices as well as storage media for temporarily and/or more permanentlycontaining, storing, transmitting and retrieving computer-readableinformation. The system and various devices also typically will includea number of software applications, modules, services or other elementslocated within at least one working memory device, including anoperating system and application programs, such as a client applicationor web browser. It should be appreciated that alternate embodiments mayhave numerous variations from that described above. For example,customized hardware might also be used and/or particular elements mightbe implemented in hardware, software (including portable software, suchas applets) or both. Further, connection to other computing devices suchas network input/output devices may be employed.

Storage media and computer readable media for containing code, orportions of code, can include any appropriate media known or used in theart, including storage media and communication media, such as, but notlimited to, volatile and non-volatile, removable and non-removable mediaimplemented in any method or technology for storage and/or transmissionof information such as computer readable instructions, data structures,program modules or other data, including RAM, ROM, Electrically ErasableProgrammable Read-Only Memory (“EEPROM”), flash memory or other memorytechnology, Compact Disc Read-Only Memory (“CD-ROM”), digital versatiledisk (DVD) or other optical storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices or any othermedium which can be used to store the desired information and which canbe accessed by the system device. Based on the disclosure and teachingsprovided herein, a person of ordinary skill in the art will appreciateother ways and/or methods to implement the various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims.

Other variations are within the spirit of the present disclosure. Thus,while the disclosed techniques are susceptible to various modificationsand alternative constructions, certain illustrated embodiments thereofare shown in the drawings and have been described above in detail. Itshould be understood, however, that there is no intention to limit theinvention to the specific form or forms disclosed, but on the contrary,the intention is to cover all modifications, alternative constructionsand equivalents falling within the spirit and scope of the invention, asdefined in the appended claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated herein or clearlycontradicted by context. The terms “comprising,” “having,” “including”and “containing” are to be construed as open-ended terms (i.e., meaning“including, but not limited to,”) unless otherwise noted. The term“connected,” when unmodified and referring to physical connections, isto be construed as partly or wholly contained within, attached to orjoined together, even if there is something intervening. Recitation ofranges of values herein are merely intended to serve as a shorthandmethod of referring individually to each separate value falling withinthe range, unless otherwise indicated herein and each separate value isincorporated into the specification as if it were individually recitedherein. The use of the term “set” (e.g., “a set of items”) or “subset”unless otherwise noted or contradicted by context, is to be construed asa nonempty collection comprising one or more members. Further, unlessotherwise noted or contradicted by context, the term “subset” of acorresponding set does not necessarily denote a proper subset of thecorresponding set, but the subset and the corresponding set may beequal.

Conjunctive language, such as phrases of the form “at least one of A, B,and C,” or “at least one of A, B and C,” unless specifically statedotherwise or otherwise clearly contradicted by context, is otherwiseunderstood with the context as used in general to present that an item,term, etc., may be either A or B or C, or any nonempty subset of the setof A and B and C. For instance, in the illustrative example of a sethaving three members used in the above conjunctive phrase, “at least oneof A, B, and C” and “at least one of A, B and C” refers to any of thefollowing sets: {A}, {B}, {C}, {A, B}, {A, C}, {B, C}, {A, B, C}. Thus,such conjunctive language is not generally intended to imply thatcertain embodiments require at least one of A, at least one of B and atleast one of C to each be present.

Operations of processes described herein can be performed in anysuitable order unless otherwise indicated herein or otherwise clearlycontradicted by context. Processes described herein (or variationsand/or combinations thereof) may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs or one or more applications) executing collectively onone or more processors, by hardware or combinations thereof. The codemay be stored on a computer-readable storage medium, for example, in theform of a computer program comprising a plurality of instructionsexecutable by one or more processors. The computer-readable storagemedium may be non-transitory.

The use of any and all examples, or exemplary language (e.g., “such as”)provided herein, is intended merely to better illuminate embodiments ofthe invention and does not pose a limitation on the scope of theinvention unless otherwise claimed. No language in the specificationshould be construed as indicating any non-claimed element as essentialto the practice of the invention.

Preferred embodiments of this disclosure are described herein, includingthe best mode known to the inventors for carrying out the invention.Variations of those preferred embodiments may become apparent to thoseof ordinary skill in the art upon reading the foregoing description. Theinventors expect skilled artisans to employ such variations asappropriate and the inventors intend for embodiments of the presentdisclosure to be practiced otherwise than as specifically describedherein. Accordingly, the scope of the present disclosure includes allmodifications and equivalents of the subject matter recited in theclaims appended hereto as permitted by applicable law. Moreover, anycombination of the above-described elements in all possible variationsthereof is encompassed by the scope of the present disclosure unlessotherwise indicated herein or otherwise clearly contradicted by context.

All references, including publications, patent applications and patents,cited herein are hereby incorporated by reference to the same extent asif each reference were individually and specifically indicated to beincorporated by reference and were set forth in its entirety herein.

What is claimed is:
 1. A computer-implemented method for processing arequest, comprising: under the control of one or more computer systemsconfigured with executable instructions, implementing, by a hypervisor,a virtual computer system instance and a controller such that: thevirtual computer system instance lacks an implementation of one or morelayers of a protocol stack used to receive the request; the controllerhas an implementation of the one or more layers of the protocol stacklacked by the virtual computer system instance; and a shared memoryregion is accessible to both the virtual computer system instance andthe controller; placing data of the request from the controller to thevirtual computer system instance into the shared memory region;providing the data from the shared memory region to the virtual computersystem; placing data for a response to the request from the virtualcomputer system to the controller into the shared memory region;providing the data for the response from the shared memory region to thecontroller; and using the implementation of the one or more layers ofthe protocol stack of the controller to provide the response to therequest.
 2. The computer-implemented method of claim 1, wherein: themethod further comprises providing, through a hypercall interface to thehypervisor accessible to both the virtual computer system instance andthe controller, a notification to a hypercall handler of the controller;and providing the data for the response from the shared memory region tothe controller as a result of the hypercall handler receiving thenotification.
 3. The computer-implemented method of claim 1, wherein:placing the data of the request into the shared memory region includeswriting, to a control area of the shared memory region, one or morelocations of the data in the control area; and providing the data fromthe shared memory region to the virtual computer system instanceincludes locating the data based at least in part on the one or morelocations written to the control area.
 4. The computer-implementedmethod of claim 1, wherein the method further comprises: generating, bythe virtual computer system instance, a first listener object using afirst object model; and generating, by the controller, a second listenerobject to receive the request, the second listener object correspondingto the first listener object.
 5. The computer-implemented method ofclaim 1, wherein the request is a hypertext transfer protocol request.6. The computer-implemented method of claim 1, wherein the methodfurther comprises: implementing, by the hypervisor, a second virtualcomputer system instance such that a second shared memory region isaccessible to both the second virtual computer system instance and thecontroller; and wherein the implementation of the one or more layers ofthe protocol stack is used by the controller to process a request forthe second virtual computer system instance.
 7. A computer-implementedmethod, comprising: under the control of one or more computer systemsconfigured with executable instructions, implementing, in a first domainof a hypervisor, a computer system instance; implementing, in a seconddomain of the hypervisor, a controller that shares access to a memoryregion with the computer system instance and that implements multiplelayers of a protocol stack including a shared layer of the protocolstack that is implemented by the computer system instance; satisfying arequest received using the protocol stack by at least: using the sharedlayer of the protocol stack and the memory region to pass response datafor the request from the computer system instance to the controller; andperforming one or more operations on the response data in the seconddomain using at least one layer, from the multiple layers of theprotocol stack, for which the computer system instance lacks animplementation.
 8. The computer-implemented method of claim 7, whereinthe request is a hypertext transfer protocol request.
 9. Thecomputer-implemented method of claim 7, wherein the shared layer is anapplication layer of the protocol stack.
 10. The computer-implementedmethod of claim 7, wherein passing the response data for the requestincludes transmitting a hypercall to the controller as a result of thecomputer system instance interacting with a paravirtual device driver toa hypercall interface of the hypervisor.
 11. The computer-implementedmethod of claim 7, wherein: the memory region is configured to comprisea control space and a plurality of request slots; passing the responsedata from the computer system instance to the controller includescausing the controller to obtain the response data by at least accessingthe control space to determine a request slot from the plurality ofrequest slots in which at least a portion of the response data islocated.
 12. The computer-implemented method of claim 7, whereinsatisfying the request includes generating a first request in the firstdomain, the first request corresponding to the request, the requesthaving been received in the second domain.
 13. A system, comprising: oneor more processors; and memory including instructions that, whenexecuted by the one or more processors, cause the system to implement: ahypervisor; one or more computer system instances operating on thehypervisor; a controller operating on the hypervisor; for each computersystem instance of the one or more computer system instances, a memoryspace shared by the computer system instance and the controller; whereinthe hypervisor, the computer system instance, and the controller arecollectively configured to satisfy a request of the computer system atleast in part by the controller by notifying, through a hypercall fromthe computer system instance to the controller, of data for a responseto the request being made available by placing the request data in thememory space.
 14. The system of claim 13, wherein the computer systeminstance includes a paravirtual device driver to a hypercall interfaceimplemented by the hypervisor such that hypercalls by one of thecomputer system instance and controller trigger correspondingcommunications to the other of the computer system instance and thecontroller.
 15. The system of claim 13, wherein: the controllercomprises an implementation of a first set of layers of a protocolsuite; the computer system instance comprises an implementation of asecond set of layers of the protocol suite; the first set of layers andthe second set of layers are different but have a nonempty intersection;and satisfying the request includes using, by the controller, the firstset of layers of the protocol suite.
 16. The system of claim 13, whereinthe request is an HTTP request.
 17. The system of claim 13, whereinplacing the request data in the memory space includes writing first datato a first portion of the memory space and writing second data to asecond portion of the memory space, the first data indicating a locationof the second data in the second portion of the memory space.
 18. Thesystem of claim 13, wherein the controller includes a hypercall handlerconfigured to receive hypercalls from a hypercall interface of thehypervisor and, as a result, access the memory space to identifyrequests to be serviced.
 19. A non-transitory computer-readable storagemedium having stored thereon instructions that, when executed by one ormore processors of a computer system, cause the computer system toimplement at least: a computer system instance in a first domain of ahypervisor; a controller in a second domain of the hypervisor; and amemory region accessible to both the controller and the computer systeminstance; wherein the computer system instance and controller arecollectively configured to cooperate in request processing such that,satisfying a request includes: using a hypercall interface of thehypervisor and the memory region to pass data between the computersystem instance and the controller; and causing both the computer systeminstance and the controller to each perform one or more operations inconnection with the data.
 20. The computer-readable storage medium ofclaim 19, wherein the request is a hypertext transfer protocol request.21. The computer-readable storage medium of claim 19, wherein passingthe data between the computer system instance and the controllerincludes the computer system instance interacting with a paravirtualdevice driver to cause a notification of the data in the memory regionto be issued to the controller.
 22. The computer-readable storage mediumof claim 21, wherein the notification includes an identifier of therequest and wherein passing the data between the computer systeminstance and the controller further includes determining a locationusing the identifier to select the request data from multiple instancesof data stored in the memory region.
 23. The computer-readable storagemedium of claim 19, wherein: the controller implements one or morelayers of a protocol stack unimplemented by the computer systeminstance; and the request processing includes use of the one or morelayers of the protocol stack.
 24. The computer-readable storage mediumof claim 23, wherein: the instructions further cause the computer systemto implement one or more other computer system instances; theimplemented one or more layers of the protocol stack is used for requestprocessing for each of the one or more other computer system instances;and the memory region is inaccessible to the one of more other computersystem instances.